GPT-3 is the 3rd release of the OpenAI collection of Generative Pre-Trained models. GPT-1 and GPT-2 laid the foundations for GPT-3, proving the success of two key hypotheses: Transformers unsupervised pre-training works fine (GPT-1), and language models can multitask (GPT-2). GPT-3 is a language model built on the transformer architecture and pre-trained in an unsupervised, generative manner which has a decent performance in one-shot, zero-shot & few-shot multitask settings. It functions by anticipating the next token in the sequence of tokens, and it can do this for NLP tasks that it's not been taught. After some instances, it reached the highest performance in specific benchmarks, like machine translating, Q&A, and Cloze tasks. GPT-3 was trained on massive Internet text databases, a total of 570GB.

Recurrent neural networks can generate text which is indistinguishable from human writing. Here is an example with Amazon product reviews data. Language modeling uses various techniques to determine the probability of a sequence of words in a sentence in a particular language.

The aim of the TextAD research project at Linköping University is to better understand different types of reading difficulties. This knowledge can be used to develop digital tools that automatically adapt texts to the needs of readers. For many people, reading can be difficult, and some people need texts adapted to their ability. This group is very diverse, and the aspects of reading that cause problems differ from one individual to another. The project will investigate how three groups aged 13-17 years (pupils with dyslexia, pupils with intellectual disabilities, and typical readers) understand texts that present factual information.